Initialization of Self-Organizing Maps: Principal Components Versus Random Initialization. A Case Study

The performance of the Self-Organizing Map (SOM) algorithm is dependent on the initial weights of the map. The different initialization methods can broadly be classified into random and data analysis based initialization approach. In this paper, the performance of random initialization (RI) approach is compared to that of principal component initialization (PCI) in which the initial map weights are chosen from the space of the principal component. Performance is evaluated by the fraction of variance unexplained (FVU). Datasets were classified into quasi-linear and non-linear and it was observed that RI performed better for non-linear datasets; however the performance of PCI approach remains inconclusive for quasi-linear datasets. Introduction Self–Organizing Map (SOM) can be considered as a non-linear generalization of the principal component analysis [14] and has found much application in data exploration especially in data visualization, vector quantization and dimension reduction. Inspired by biological neural networks, it is a type of artificial neural network which uses an unsupervised learning algorithm with the additional property that it preserves the topological mapping from input space to output space making it a great tool for visualization of high dimensional data in a lower dimension. Originally developed for visualization of distribution of metric vectors [12], SOM found early application in speech recognition. However, like clustering algorithms, the quality of learning of SOM is greatly influenced by the initial conditions: initial weight of the map, the neighbourhood function, the learning rate, sequence of training vector and number of iterations. [1][12][11]. Several initialization approaches have been developed and can be broadly grouped into two classes: random initialization and data analysis based initialization [1]. Due to many possible initial configurations when using random approach, several attempts are usually made and the best initial configuration is adopted. However, for the data analysis based approach, certain statistical data analysis and data classification methods are used to determine the initial configuration; a popular method is selecting the initial weights from the same space spanned by the linear principal component (first eigenvectors corresponding to the largest eigenvalues of the empirical covariance matrix). Modification to the PCA approach was done by [1] and over the years other initialization methods have been proposed. An example is given by [4]. In this paper we consider the performance in terms of the quality of learning of the SOM using the random initialization (RI) method (in which the initial weight is taking from the sample data) and the principal component initialization (PCI) method. The quality of learning is determined by the fraction of variance unexplained [8]. To ensure an exhaustive study, synthetic data sets distributed along various shapes of only 2-dimensions are considered in this study and the map is 1-dimensional. 1 Dimension SOM is very important, for example, for approximation of principal curves. The experiment was performed using the PCA, SOM and GSOM applet available online [8].